Methods used to develop the SPIRIT 2024 and CONSORT 2024 Statements.

OBJECTIVES: To describe, and explain the rationale for, the methods used and decisions made during development of the updated SPIRIT 2024 and CONSORT 2024 reporting guidelines. METHODS: We developed SPIRIT 2024 and CONSORT 2024 together to facilitate harmonization of the two guidelines, and incorporated content from key extensions. We conducted a scoping review of comments suggesting changes to SPIRIT 2013 and CONSORT 2010, and compiled a list of other possible revisions based on existing SPIRIT and CONSORT extensions, other reporting guidelines, and personal communications. From this, we generated a list of potential modifications or additions to SPIRIT and CONSORT, which we presented to stakeholders for feedback in an international online Delphi survey. The Delphi survey results were discussed at an online expert consensus meeting attended by 30 invited international participants. We then drafted the updated SPIRIT and CONSORT checklists and revised them based on further feedback from meeting attendees. RESULTS: We compiled 83 suggestions for revisions or additions to SPIRIT and/or CONSORT from the scoping review and 85 from other sources, from which we generated 33 potential changes to SPIRIT (n = 5) or CONSORT (n = 28). Of 463 participants invited to take part in the Delphi survey, 317 (68%) responded to Round 1, 303 (65%) to Round 2 and 290 (63%) to Round 3. Two additional potential checklist changes were added to the Delphi survey based on Round 1 comments. Overall, 14/35 (SPIRIT n = 0; CONSORT n = 14) proposed changes reached the predefined consensus threshold (≥80% agreement), and participants provided 3580 free-text comments. The consensus meeting participants agreed with implementing 11/14 of the proposed changes that reached consensus in the Delphi and supported implementing a further 4/21 changes (SPIRIT n = 2; CONSORT n = 2) that had not reached the Delphi threshold. They also recommended further changes to refine key concepts and for clarity. CONCLUSION: The forthcoming SPIRIT 2024 and CONSORT 2024 Statements will provide updated, harmonized guidance for reporting randomized controlled trial protocols and results, respectively. The simultaneous development of the SPIRIT and CONSORT checklists has been informed by current empirical evidence and extensive input from stakeholders. We hope that this report of the methods used will be helpful for developers of future reporting guidelines.

A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E).

BACKGROUND: Observational epidemiologic studies provide critical data for the evaluation of the potential effects of environmental, occupational and behavioural exposures on human health. Systematic reviews of these studies play a key role in informing policy and practice. Systematic reviews should incorporate assessments of the risk of bias in results of the included studies. OBJECTIVE: To develop a new tool, Risk Of Bias In Non-randomized Studies - of Exposures (ROBINS-E) to assess risk of bias in estimates from cohort studies of the causal effect of an exposure on an outcome. METHODS AND RESULTS: ROBINS-E was developed by a large group of researchers from diverse research and public health disciplines through a series of working groups, in-person meetings and pilot testing phases. The tool aims to assess the risk of bias in a specific result (exposure effect estimate) from an individual observational study that examines the effect of an exposure on an outcome. A series of preliminary considerations informs the core ROBINS-E assessment, including details of the result being assessed and the causal effect being estimated. The assessment addresses bias within seven domains, through a series of 'signalling questions'. Domain-level judgements about risk of bias are derived from the answers to these questions, then combined to produce an overall risk of bias judgement for the result, together with judgements about the direction of bias. CONCLUSION: ROBINS-E provides a standardized framework for examining potential biases in results from cohort studies. Future work will produce variants of the tool for other epidemiologic study designs (e.g. case-control studies). We believe that ROBINS-E represents an important development in the integration of exposure assessment, evidence synthesis and causal inference.

Transitivity, coherence, and reliability of network meta-analyses comparing proximal humerus fracture treatments: a meta-epidemiological study.

BACKGROUND: Network meta-analyses can be valuable for decision-makers in guiding clinical practice. However, for network meta-analysis results to be reliable, the assumptions of both transitivity and coherence must be met, and the methodology should adhere to current best practices. We aimed to assess whether network meta-analyses of randomized controlled trials (RCTs) comparing interventions for proximal humerus fractures provide reliable estimates of intervention effects. METHODS: We searched PubMed, EMBASE, The Cochrane Library, and Web of Science for network meta-analyses comparing interventions for proximal humerus fractures. We critically assessed the methodology regarding the development of a protocol, search strategy, trial inclusion, outcome extraction, and the methods used to conduct the network meta-analyses. We assessed the transitivity and coherence of the network graphs for the Constant score (CS), Disabilities of the Arm, Shoulder, and Hand score (DASH), and additional surgery. Transitivity was assessed by comparing probable effect modifiers (age, gender, fracture morphology, and comorbidities) across intervention comparisons. Coherence was assessed using Separating Indirect from Direct Evidence (SIDE) (Separating Indirect from Direct Evidence) and the design-by-treatment interaction test. We used CINeMA (Confidence in Network Meta-analyses) to assess the confidence in the results. RESULTS: None of the three included network meta-analyses had a publicly available protocol or data-analysis plan, and they all had methodological flaws that could threaten the validity of their results. Although we did not detect incoherence for most comparisons, the transitivity assumption was violated for CS, DASH, and additional surgery in all three network meta-analyses. Additionally, the confidence in the results was 'very low' primarily due to within-study bias, reporting bias, intransitivity, imprecision, and heterogeneity. CONCLUSIONS: Current network meta-analyses of RCTs comparing interventions for proximal humerus fractures do not provide reliable estimates of intervention effects. We advise caution in using these network meta-analyses to guide clinical practice. To improve the utility of network meta-analyses to guide clinical practice, journal editors should require that network meta-analyses are done according to a predefined analysis plan in a publicly available protocol and that both coherence and transitivity have been adequately assessed and reported.

A structured approach to information retrieval improved identification of funding and researchers' conflicts of interest in trials included in Cochrane reviews.

OBJECTIVES: To compare the contemporary Cochrane review approach for retrieving information on trial funding and researchers' conflicts of interest with a structured approach for information retrieval. STUDY DESIGN AND SETTING: Methodological study of 100 Cochrane reviews from August to December 2020 and one randomly selected trial from each review. Reporting of trial funding and researchers' conflicts of interest in reviews was compared with information identified using a structured retrieval process, and time to retrieve information was noted. We also formulated a guide to systematic reviewers for efficient information retrieval. RESULTS: Sixty-eight of 100 Cochrane reviews reported trial funding and 24 reported trial researchers' conflicts of interest. A simple structured approach, searching only trial publications (including conflicts of interest disclosure forms), identified funding for 16 additional trials and conflicts of interest information for 39 additional trials. A comprehensive structured approach, searching multiple information sources, identified funding for two additional trials and conflicts of interest for 14 additional trials. The median time to retrieve information was 10 minutes per trial (interquartile range: 7-15) for the simple approach and 20 minutes (11-43) for the comprehensive approach. CONCLUSION: A structured information retrieval approach improves identification of funding and researchers' conflicts of interest in trials included in Cochrane reviews.

Interleukin-6 blocking agents for treating COVID-19: a living systematic review.

BACKGROUND: It has been reported that people with COVID-19 and pre-existing autoantibodies against type I interferons are likely to develop an inflammatory cytokine storm responsible for severe respiratory symptoms. Since interleukin 6 (IL-6) is one of the cytokines released during this inflammatory process, IL-6 blocking agents have been used for treating people with severe COVID-19. OBJECTIVES: To update the evidence on the effectiveness and safety of IL-6 blocking agents compared to standard care alone or to a placebo for people with COVID-19. SEARCH METHODS: We searched the World Health Organization (WHO) International Clinical Trials Registry Platform, the Living OVerview of Evidence (L·OVE) platform, and the Cochrane COVID-19 Study Register to identify studies on 7 June 2022. SELECTION CRITERIA: We included randomized controlled trials (RCTs) evaluating IL-6 blocking agents compared to standard care alone or to placebo for people with COVID-19, regardless of disease severity. DATA COLLECTION AND ANALYSIS: Pairs of researchers independently conducted study selection, extracted data and assessed risk of bias. We assessed the certainty of evidence using the GRADE approach for all critical and important outcomes. In this update we amended our protocol to update the methods used for grading evidence by establishing minimal important differences for the critical outcomes. MAIN RESULTS: This update includes 22 additional trials, for a total of 32 trials including 12,160 randomized participants all hospitalized for COVID-19 disease. We identified a further 17 registered RCTs evaluating IL-6 blocking agents without results available as of 7 June 2022.  The mean age range varied from 56 to 75 years; 66.2% (8051/12,160) of enrolled participants were men. One-third (11/32) of included trials were placebo-controlled. Twenty-two were published in peer-reviewed journals, three were reported as preprints, two trials had results posted only on registries, and results from five trials were retrieved from another meta-analysis. Eight were funded by pharmaceutical companies.  Twenty-six included studies were multicenter trials; four were multinational and 22 took place in single countries. Recruitment of participants occurred between February 2020 and June 2021, with a mean enrollment duration of 21 weeks (range 1 to 54 weeks). Nineteen trials (60%) had a follow-up of 60 days or more. Disease severity ranged from mild to critical disease. The proportion of participants who were intubated at study inclusion also varied from 5% to 95%. Only six trials reported vaccination status; there were no vaccinated participants included in these trials, and 17 trials were conducted before vaccination was rolled out. We assessed a total of six treatments, each compared to placebo or standard care. Twenty trials assessed tocilizumab, nine assessed sarilumab, and two assessed clazakizumab. Only one trial was included for each of the other IL-6 blocking agents (siltuximab, olokizumab, and levilimab). Two trials assessed more than one treatment. Efficacy and safety of tocilizumab and sarilumab compared to standard care or placebo for treating COVID-19 At day (D) 28, tocilizumab and sarilumab probably result in little or no increase in clinical improvement (tocilizumab: risk ratio (RR) 1.05, 95% confidence interval (CI) 1.00 to 1.11; 15 RCTs, 6116 participants; moderate-certainty evidence; sarilumab: RR 0.99, 95% CI 0.94 to 1.05; 7 RCTs, 2425 participants; moderate-certainty evidence). For clinical improvement at ≥ D60, the certainty of evidence is very low for both tocilizumab (RR 1.10, 95% CI 0.81 to 1.48; 1 RCT, 97 participants; very low-certainty evidence) and sarilumab (RR 1.22, 95% CI 0.91 to 1.63; 2 RCTs, 239 participants; very low-certainty evidence). The effect of tocilizumab on the proportion of participants with a WHO Clinical Progression Score (WHO-CPS) of level 7 or above remains uncertain at D28 (RR 0.90, 95% CI 0.72 to 1.12; 13 RCTs, 2117 participants; low-certainty evidence) and that for sarilumab very uncertain (RR 1.10, 95% CI 0.90 to 1.33; 5 RCTs, 886 participants; very low-certainty evidence). Tocilizumab reduces all cause-mortality at D28 compared to standard care/placebo (RR 0.88, 95% CI 0.81 to 0.94; 18 RCTs, 7428 participants; high-certainty evidence). The evidence about the effect of sarilumab on this outcome is very uncertain (RR 1.06, 95% CI 0.86 to 1.30; 9 RCTs, 3305 participants; very low-certainty evidence). The evidence is uncertain for all cause-mortality at ≥ D60 for tocilizumab (RR 0.91, 95% CI 0.80 to 1.04; 9 RCTs, 2775 participants; low-certainty evidence) and very uncertain for sarilumab (RR 0.95, 95% CI 0.84 to 1.07; 6 RCTs, 3379 participants; very low-certainty evidence). Tocilizumab probably results in little to no difference in the risk of adverse events (RR 1.03, 95% CI 0.95 to 1.12; 9 RCTs, 1811 participants; moderate-certainty evidence). The evidence about adverse events for sarilumab is uncertain (RR 1.12, 95% CI 0.97 to 1.28; 4 RCT, 860 participants; low-certainty evidence).  The evidence about serious adverse events is very uncertain for tocilizumab (RR 0.93, 95% CI 0.81 to 1.07; 16 RCTs; 2974 participants; very low-certainty evidence) and uncertain for sarilumab (RR 1.09, 95% CI 0.97 to 1.21; 6 RCTs; 2936 participants; low-certainty evidence). Efficacy and safety of clazakizumab, olokizumab, siltuximab and levilimab compared to standard care or placebo for treating COVID-19 The evidence about the effects of clazakizumab, olokizumab, siltuximab, and levilimab comes from only one or two studies for each blocking agent, and is uncertain or very uncertain. AUTHORS' CONCLUSIONS: In hospitalized people with COVID-19, results show a beneficial effect of tocilizumab on all-cause mortality in the short term and probably little or no difference in the risk of adverse events compared to standard care alone or placebo. Nevertheless, both tocilizumab and sarilumab probably result in little or no increase in clinical improvement at D28. Evidence for an effect of sarilumab and the other IL-6 blocking agents on critical outcomes is uncertain or very uncertain. Most of the trials included in our review were done before the waves of different variants of concern and before vaccination was rolled out on a large scale. An additional 17 RCTs of IL-6 blocking agents are currently registered with no results yet reported. The number of pending studies and the number of participants planned is low. Consequently, we will not publish further updates of this review.

Meta-epidemiology and the MetaBLIND study: Response to Tack.

In 2020 we published MetaBLIND, a large meta-epidemiological study on the impact of masking on effect estimates in randomized clinical trials (RCTs). While masking is an established methodological practice in RCTs it is not clear to what extent results of non-masked RCTs are biased. Surprisingly, we found no evidence of an impact of masking on effect estimates, on average. Michiel Tack commented on the MetaBLIND study, and here we respond. The issues he raised were examples of standard themes when interpreting meta-epidemiological studies, which we have discussed at some length elsewhere, and did not warrant change of our conclusion. We maintain that the MetaBLIND results do not provide a sufficient basis for recommending abandoning masking as a methodological safeguard.

Impact of active placebo controls on estimated drug effects in randomised trials: a systematic review of trials with both active placebo and standard placebo.

BACKGROUND: An estimated 60% of pharmacological randomised trials use placebo control interventions to blind (i.e. mask) participants. However, standard placebos do not control for perceptible non-therapeutic effects (i.e. side effects) of the experimental drug, which may unblind participants. Trials rarely use active placebo controls, which contain pharmacological compounds designed to mimic the non-therapeutic experimental drug effects in order to reduce the risk of unblinding. A relevant improvement in the estimated effects of active placebo compared with standard placebo would imply that trials with standard placebo may overestimate experimental drug effects. OBJECTIVES: We aimed to estimate the difference in drug effects when an experimental drug is compared with an active placebo versus a standard placebo control intervention, and to explore causes for heterogeneity. In the context of a randomised trial, this difference in drug effects can be estimated by directly comparing the effect difference between the active placebo and standard placebo intervention. SEARCH METHODS: We searched PubMed, CENTRAL, Embase, two other databases, and two trial registries up to October 2020. We also searched reference lists and citations and contacted trial authors. SELECTION CRITERIA: We included randomised trials that compared an active placebo versus a standard placebo intervention. We considered trials both with and without a matching experimental drug arm. DATA COLLECTION AND ANALYSIS: We extracted data, assessed risk of bias, scored active placebos for adequacy and risk of unintended therapeutic effect, and categorised active placebos as unpleasant, neutral, or pleasant. We requested individual participant data from the authors of four cross-over trials published after 1990 and one unpublished trial registered after 1990. Our primary inverse-variance, random-effects meta-analysis used standardised mean differences (SMDs) of active versus standard placebo for participant-reported outcomes at earliest post-treatment assessment. A negative SMD favoured the active placebo. We stratified analyses by trial type (clinical or preclinical) and supplemented with sensitivity and subgroup analyses and meta-regression. In secondary analyses, we investigated observer-reported outcomes, harms, attrition, and co-intervention outcomes. MAIN RESULTS: We included 21 trials (1462 participants). We obtained individual participant data from four trials. Our primary analysis of participant-reported outcomes at earliest post-treatment assessment resulted in a pooled SMD of -0.08 (95% confidence interval (CI) -0.20 to 0.04; I2 = 31%; 14 trials), with no clear difference between clinical and preclinical trials. Individual participant data contributed 43% of the weight of this analysis. Two of seven sensitivity analyses found more pronounced and statistically significant differences; for example, in the five trials with low overall risk of bias, the pooled SMD was -0.24 (95% CI -0.34 to -0.13). The pooled SMD of observer-reported outcomes was similar to the primary analysis. The pooled odds ratio (OR) for harms was 3.08 (95% CI 1.56 to 6.07), and for attrition, 1.22 (95% CI 0.74 to 2.03). Co-intervention data were limited. Meta-regression found no statistically significant association with adequacy of the active placebo or risk of unintended therapeutic effect. AUTHORS' CONCLUSIONS: We did not find a statistically significant difference between active and standard placebo control interventions in our primary analysis, but the result was imprecise and the CI compatible with a difference ranging from important to irrelevant. Furthermore, the result was not robust, because two sensitivity analyses produced a more pronounced and statistically significant difference. We suggest that trialists and users of information from trials carefully consider the type of placebo control intervention in trials with high risk of unblinding, such as those with pronounced non-therapeutic effects and participant-reported outcomes.

A meta-epidemiological study found that meta-analyses of the same trials may obtain drastically conflicting results.

OBJECTIVES: To assess how much protocols reduce methodological flexibility and variation of results in meta-analyses comparing operative with nonoperative treatments for proximal humerus fractures. STUDY DESIGN AND SETTING: A meta-epidemiological study. We searched four bibliographic databases for eligible meta-analyses. We contacted the authors of the meta-analyses, requesting a copy of their protocol. We identified the included studies and extracted intervention effect data for functional outcome, quality of life, and adverse events. Using the same intervention effect data for each outcome domain, we conducted multiple meta-analyses using differing combinations of methodological choices (analytical scenarios) without restricting the available methodological choices (as if no protocol was used). For each protocol, we repeated this process but restricted the available choices to those listed in the protocol. We then compared the distributions of effect estimates with and without protocols. Methodological flexibility was estimated as the difference in number of possible meta-analyses and the variation of results as the difference between the most conflicting effect estimates. RESULTS: We identified 23 meta-analyses, included 24 primary studies, and obtained three protocols. The protocols markedly reduced methodological flexibility (range for functional outcomes 94%-99%; quality of life 58%-76%; adverse events 87%-91%). The corresponding reduction in the variation of the results varied (range for functional outcomes; 33%-78%, quality of life; 10%-24%; adverse events 10%-13%). CONCLUSION: Although the protocols consistently reduced methodological flexibility, the reduction in the variation (bias) of results varied. Consequently, review authors may obtain conflicting results based on the same data. We advise caution when interpreting the conclusions of meta-analyses with absent or imprecise protocols.

A scoping review identifies multiple comments suggesting modifications to SPIRIT 2013 and CONSORT 2010.

OBJECTIVES: To identify, summarize, and analyse comments on the core reporting guidelines for protocols of randomized trials (Standard Protocol Items: Recommendations for Interventional Trials [SPIRIT] 2013) and for completed trials (Consolidated Standards of Reporting Trials [CONSORT] 2010), with special emphasis on suggestions for guideline modifications. METHODS: We included documents written in English and published after 2010 that explicitly commented on SPIRIT 2013 or CONSORT 2010. We searched four bibliographic databases (Embase and MEDLINE to June 2022; Web of Science and Google Scholar to April 2022) and other sources (e.g., the EQUATOR Network website, the BMC Blog Network, and the BMJ rapid response section). Two authors independently assessed documents for eligibility and extracted data on basic characteristics and the wording of the main comments. We categorized comments as 'suggestion for modification to the wording of an existing guideline item,' 'suggestion for a new item,' or 'reflections on challenges or strengths.' We provided a summary and examples of the proposed suggestions and categorized comments into those that were directly linked to empirical investigations, were continuations of previous methodological discussions, or reflected new methodological developments. RESULTS: We assessed full text of 2,320 potentially eligible documents and included 93 documents with 114 comments. In total, 37 comments suggested modifications to existing guideline items. The participant flow section of CONSORT 2010 received the most comments (eight comments made different suggestions, e.g., one comment suggested to add numbers on nonrandomized screened participants). There were 46 comments suggesting new items. Multiple suggestions were related to trial interventions (eight comments made different suggestions, e.g., one comment suggested to add content on cointerventions), blinding (six comments suggested to add content on risk of unblinding), statistical methods (five comments made different suggestions, e.g., one comment suggested to add content on blinding of statisticians), and participant flow (seven comments made different suggestions, e.g., three comments suggested to add content on missing data). Half (53%) of the suggestions were directly linked to empirical investigations. Six (7%) suggestions were continuations of previous methodological discussions and five (6%) suggestions reflected new methodological developments related to conflicts of interest and funding, data sharing, and patient and public involvement. CONCLUSION: The issues raised provide context to authors, peer reviewers, editors, and readers of trials using SPIRIT 2013 and CONSORT 2010 and inform the planned updates of the core guidelines.

Using Risk of Bias 2 to assess results from randomised controlled trials: guidance from Cochrane.

A systematic review identifies, appraises and synthesises all the empirical evidence from studies that meet prespecified eligibility criteria to answer a specific research question. As part of the appraisal, researchers use explicit methods to assess risk of bias in the results' from included studies that contribute to the review's findings, to improve our confidence in the review's conclusions. Randomised controlled trials included in Cochrane Reviews have used a specific risk of bias tool to assess these included studies since 2008. In 2019, a new version of this tool, Risk of Bias 2 (RoB 2), was launched to improve its usability and to reflect current understanding of how the causes of bias can influence study results. Cochrane implemented RoB 2 in a phased approach, with users of the tool informing guidance development. This paper highlights learning for all systematic reviewers (Cochrane and non-Cochrane) from the phased implementation, highlighting differences between the original version of the tool and RoB 2, consideration of reporting systematic review protocols or full review reports that have used RoB 2, and some tips shared by authors during the pilot phase of the implementation.

A scoping review finds that guides to authors of protocols for observational epidemiological studies varied highly in format and content.

OBJECTIVE: To identify, characterize, and explore author guides on the role, format, and content of protocols for observational epidemiological studies, particularly cohort and case-control studies. STUDY DESIGN AND SETTING: Scoping review. We searched for guides in Medline, Embase, Google Scholar, 10 general medical and epidemiological/public health journals, and 10 major funders' websites. Two review authors extracted data. We classified guides as "main" based on word count and number of protocol items, described such guides more comprehensively and analyzed number of citations as an indicator of uptake. RESULTS: Thirty-nine protocol guides were included intended for cohort studies (n = 3), case-control studies (n = 1), or epidemiological studies in general (n = 35). Content and format were highly variable. Several guides had a broader focus than protocol development, e.g., also including study conduct and reporting. The guideline developmental process was often reported sparsely. One guide, intended for interventional studies, combined a systematic preparatory process with a primary focus on protocol development. We categorized seven guides as 'main'. In general the guides were cited infrequently, indicating limited uptake. CONCLUSION: Guides for authors of protocols for observational epidemiological studies varied highly in format and content. We suggest that such guides should routinely be based on a systematic preparatory process.

Commercial funding and estimated intervention effects in randomized clinical trials: Systematic review of meta-epidemiological studies.

We investigated to which degree commercial funding is associated with estimated intervention effects in randomized trials. We included meta-epidemiological studies with published data on the association between commercial funding and results or conclusions of randomized trials. We searched five databases and other sources. We selected one result per meta-epidemiological study, preferably unadjusted ratio of odds ratios (ROR), for example, odds ratio(commercial funding)/odds ratio(noncommercial funding). We pooled RORs in random-effects meta-analyses (ROR <1 indicated exaggerated intervention effects in commercially funded trials), subgrouped (preplanned) by study aim: commercial funding per se versus risk of commercial funder influence. We included eight meta-epidemiological studies (264 meta-analyses, 2725 trials). The summary ROR was 0.95 (95% confidence interval 0.85-1.06). Subgroup analysis revealed a difference (p = 0.02) between studies of commercial funding per se, ROR 1.06 (0.95-1.17) and studies of risk of commercial funder influence, ROR 0.88 (0.79-0.97). In conclusion, we found no statistically significant association between commercial funding and estimated intervention effects when combining studies of commercial funding per se and studies of risk of commercial funder influence. A preplanned subgroup analysis indicated that trials with high risk of commercial funder influence exaggerated intervention effects by 12% (21%-3%), on average. Our results differ from previous theoretical considerations and findings from methodological studies and therefore call for confirmation. We suggest it is prudent to interpret results from commercially funded trials with caution, especially when there is a risk that the funder had direct influence on trial design, conduct, analysis, or reporting.

Efficacy and safety of COVID-19 vaccines.

BACKGROUND: Different forms of vaccines have been developed to prevent the SARS-CoV-2 virus and subsequent COVID-19 disease. Several are in widespread use globally.  OBJECTIVES: To assess the efficacy and safety of COVID-19 vaccines (as a full primary vaccination series or a booster dose) against SARS-CoV-2. SEARCH METHODS: We searched the Cochrane COVID-19 Study Register and the COVID-19 L·OVE platform (last search date 5 November 2021). We also searched the WHO International Clinical Trials Registry Platform, regulatory agency websites, and Retraction Watch. SELECTION CRITERIA: We included randomized controlled trials (RCTs) comparing COVID-19 vaccines to placebo, no vaccine, other active vaccines, or other vaccine schedules. DATA COLLECTION AND ANALYSIS: We used standard Cochrane methods. We used GRADE to assess the certainty of evidence for all except immunogenicity outcomes.  We synthesized data for each vaccine separately and presented summary effect estimates with 95% confidence intervals (CIs).  MAIN RESULTS: We included and analyzed 41 RCTs assessing 12 different vaccines, including homologous and heterologous vaccine schedules and the effect of booster doses. Thirty-two RCTs were multicentre and five were multinational. The sample sizes of RCTs were 60 to 44,325 participants. Participants were aged: 18 years or older in 36 RCTs; 12 years or older in one RCT; 12 to 17 years in two RCTs; and three to 17 years in two RCTs. Twenty-nine RCTs provided results for individuals aged over 60 years, and three RCTs included immunocompromized patients. No trials included pregnant women. Sixteen RCTs had two-month follow-up or less, 20 RCTs had two to six months, and five RCTs had greater than six to 12 months or less. Eighteen reports were based on preplanned interim analyses. Overall risk of bias was low for all outcomes in eight RCTs, while 33 had concerns for at least one outcome. We identified 343 registered RCTs with results not yet available.  This abstract reports results for the critical outcomes of confirmed symptomatic COVID-19, severe and critical COVID-19, and serious adverse events only for the 10 WHO-approved vaccines. For remaining outcomes and vaccines, see main text. The evidence for mortality was generally sparse and of low or very low certainty for all WHO-approved vaccines, except AD26.COV2.S (Janssen), which probably reduces the risk of all-cause mortality (risk ratio (RR) 0.25, 95% CI 0.09 to 0.67; 1 RCT, 43,783 participants; high-certainty evidence). Confirmed symptomatic COVID-19 High-certainty evidence found that BNT162b2 (BioNtech/Fosun Pharma/Pfizer), mRNA-1273 (ModernaTx), ChAdOx1 (Oxford/AstraZeneca), Ad26.COV2.S, BBIBP-CorV (Sinopharm-Beijing), and BBV152 (Bharat Biotect) reduce the incidence of symptomatic COVID-19 compared to placebo (vaccine efficacy (VE): BNT162b2: 97.84%, 95% CI 44.25% to 99.92%; 2 RCTs, 44,077 participants; mRNA-1273: 93.20%, 95% CI 91.06% to 94.83%; 2 RCTs, 31,632 participants; ChAdOx1: 70.23%, 95% CI 62.10% to 76.62%; 2 RCTs, 43,390 participants; Ad26.COV2.S: 66.90%, 95% CI 59.10% to 73.40%; 1 RCT, 39,058 participants; BBIBP-CorV: 78.10%, 95% CI 64.80% to 86.30%; 1 RCT, 25,463 participants; BBV152: 77.80%, 95% CI 65.20% to 86.40%; 1 RCT, 16,973 participants). Moderate-certainty evidence found that NVX-CoV2373 (Novavax) probably reduces the incidence of symptomatic COVID-19 compared to placebo (VE 82.91%, 95% CI 50.49% to 94.10%; 3 RCTs, 42,175 participants). There is low-certainty evidence for CoronaVac (Sinovac) for this outcome (VE 69.81%, 95% CI 12.27% to 89.61%; 2 RCTs, 19,852 participants). Severe or critical COVID-19 High-certainty evidence found that BNT162b2, mRNA-1273, Ad26.COV2.S, and BBV152 result in a large reduction in incidence of severe or critical disease due to COVID-19 compared to placebo (VE: BNT162b2: 95.70%, 95% CI 73.90% to 99.90%; 1 RCT, 46,077 participants; mRNA-1273: 98.20%, 95% CI 92.80% to 99.60%; 1 RCT, 28,451 participants; AD26.COV2.S: 76.30%, 95% CI 57.90% to 87.50%; 1 RCT, 39,058 participants; BBV152: 93.40%, 95% CI 57.10% to 99.80%; 1 RCT, 16,976 participants). Moderate-certainty evidence found that NVX-CoV2373 probably reduces the incidence of severe or critical COVID-19 (VE 100.00%, 95% CI 86.99% to 100.00%; 1 RCT, 25,452 participants). Two trials reported high efficacy of CoronaVac for severe or critical disease with wide CIs, but these results could not be pooled. Serious adverse events (SAEs) mRNA-1273, ChAdOx1 (Oxford-AstraZeneca)/SII-ChAdOx1 (Serum Institute of India), Ad26.COV2.S, and BBV152 probably result in little or no difference in SAEs compared to placebo (RR: mRNA-1273: 0.92, 95% CI 0.78 to 1.08; 2 RCTs, 34,072 participants; ChAdOx1/SII-ChAdOx1: 0.88, 95% CI 0.72 to 1.07; 7 RCTs, 58,182 participants; Ad26.COV2.S: 0.92, 95% CI 0.69 to 1.22; 1 RCT, 43,783 participants); BBV152: 0.65, 95% CI 0.43 to 0.97; 1 RCT, 25,928 participants). In each of these, the likely absolute difference in effects was fewer than 5/1000 participants. Evidence for SAEs is uncertain for BNT162b2, CoronaVac, BBIBP-CorV, and NVX-CoV2373 compared to placebo (RR: BNT162b2: 1.30, 95% CI 0.55 to 3.07; 2 RCTs, 46,107 participants; CoronaVac: 0.97, 95% CI 0.62 to 1.51; 4 RCTs, 23,139 participants; BBIBP-CorV: 0.76, 95% CI 0.54 to 1.06; 1 RCT, 26,924 participants; NVX-CoV2373: 0.92, 95% CI 0.74 to 1.14; 4 RCTs, 38,802 participants). For the evaluation of heterologous schedules, booster doses, and efficacy against variants of concern, see main text of review. AUTHORS' CONCLUSIONS: Compared to placebo, most vaccines reduce, or likely reduce, the proportion of participants with confirmed symptomatic COVID-19, and for some, there is high-certainty evidence that they reduce severe or critical disease. There is probably little or no difference between most vaccines and placebo for serious adverse events. Over 300 registered RCTs are evaluating the efficacy of COVID-19 vaccines, and this review is updated regularly on the COVID-NMA platform (covid-nma.com). Implications for practice Due to the trial exclusions, these results cannot be generalized to pregnant women, individuals with a history of SARS-CoV-2 infection, or immunocompromized people. Most trials had a short follow-up and were conducted before the emergence of variants of concern. Implications for research Future research should evaluate the long-term effect of vaccines, compare different vaccines and vaccine schedules, assess vaccine efficacy and safety in specific populations, and include outcomes such as preventing long COVID-19. Ongoing evaluation of vaccine efficacy and effectiveness against emerging variants of concern is also vital.

Adherence to the PRISMA-P 2015 reporting guideline was inadequate in systematic review protocols.

OBJECTIVES: The objective of the study was to investigate to which degree systematic review protocols adhere to the Preferred Reporting Items for Systematic review and Meta-Analysis Protocols (PRISMA-P) reporting guideline. STUDY DESIGN AND SETTING: We randomly sampled 50 publications of systematic review protocols indexed in PubMed and 50 protocols uploaded to the International Prospective Register of Systematic Reviews (PROSPERO) from 2016 onward. Two authors independently extracted data and assessed adherence to the 26 items specified by PRISMA-P. For each protocol, we categorized adherence to PRISMA-P as complete (≥90% of PRISMA-P items were fully reported) or partial (≥60% of PRISMA-P items were fully reported). We also assessed adherence to each PRISMA-P item across the protocols. RESULTS: Four (8%) of the PubMed-indexed protocols adhered completely and 45 (90%) adhered partially to PRISMA-P but with considerable variation. None (0%) of the PROSPERO-uploaded protocols adhered completely and only 6 (12%) adhered partially to PRISMA-P. For both types of protocols, aspects related to the role of the sponsor, procedures for doing qualitative data synthesis if quantitative synthesis is not appropriate, and methods for assessing publication or outcome reporting biases and confidence in cumulative evidence were often not reported. CONCLUSION: Adherence to the PRISMA-P reporting guideline was somewhat inadequate in PubMed-indexed protocols and clearly inadequate in PROSPERO-uploaded protocols. Authors of systematic review protocols who decide to report according to PRISMA-P should carefully check all items included in the guideline, and journal editors and peer reviewers should consider PRISMA-P adherence when reviewing protocols for potential publication.

Control interventions in randomised trials among people with mental health disorders.

BACKGROUND: Control interventions in randomised trials provide a frame of reference for the experimental interventions and enable estimations of causality. In the case of randomised trials assessing patients with mental health disorders, many different control interventions are used, and the choice of control intervention may have considerable impact on the estimated effects of the treatments being evaluated. OBJECTIVES: To assess the benefits and harms of typical control interventions in randomised trials with patients with mental health disorders. The difference in effects between control interventions translates directly to the impact a control group has on the estimated effect of an experimental intervention. We aimed primarily to assess the difference in effects between (i) wait-list versus no-treatment, (ii) usual care versus wait-list or no-treatment, and (iii) placebo interventions (all placebo interventions combined or psychological, pharmacological, and physical placebos individually) versus wait-list or no-treatment. Wait-list patients are offered the experimental intervention by the researchers after the trial has been finalised if it offers more benefits than harms, while no-treatment participants are not offered the experimental intervention by the researchers. SEARCH METHODS: In March 2018, we searched MEDLINE, PsycInfo, Embase, CENTRAL, and seven other databases and six trials registers. SELECTION CRITERIA: We included randomised trials assessing patients with a mental health disorder that compared wait-list, usual care, or placebo interventions with wait-list or no-treatment . DATA COLLECTION AND ANALYSIS: Titles, abstracts, and full texts were reviewed for eligibility. Review authors independently extracted data and assessed risk of bias using Cochrane's risk of bias tool. GRADE was used to assess the quality of the evidence. We contacted researchers working in the field to ask for data from additional published and unpublished trials. A pre-planned decision hierarchy was used to select one benefit and one harm outcome from each trial. For the assessment of benefits, we summarised continuous data as standardised mean differences (SMDs) and dichotomous data as risk ratios (RRs). We used risk differences (RDs) for the assessment of adverse events. We used random-effects models for all statistical analyses. We used subgroup analysis to explore potential causes for heterogeneity (e.g. type of placebo) and sensitivity analyses to explore the robustness of the primary analyses (e.g. fixed-effect model). MAIN RESULTS: We included 96 randomised trials (4200 participants), ranging from 8 to 393 participants in each trial. 83 trials (3614 participants) provided usable data. The trials included 15 different mental health disorders, the most common being anxiety (25 trials), depression (16 trials), and sleep-wake disorders (11 trials). All 96 trials were assessed as high risk of bias partly because of the inability to blind participants and personnel in trials with two control interventions. The quality of evidence was rated low to very low, mostly due to risk of bias, imprecision in estimates, and heterogeneity. Only one trial compared wait-list versus no-treatment directly but the authors were not able to provide us with any usable data on the comparison. Five trials compared usual care versus wait-list or no-treatment and found a SMD -0.33 (95% CI -0.83 to 0.16, I² = 86%, 523 participants) on benefits. The difference between all placebo interventions combined versus wait-list or no-treatment was SMD -0.37 (95% CI -0.49 to -0.25, I² = 41%, 65 trials, 2446 participants) on benefits. There was evidence of some asymmetry in the funnel plot (Egger's test P value of 0.087). Almost all the trials were small. Subgroup analysis found a moderate effect in favour of psychological placebos SMD -0.49 (95% CI -0.64 to -0.30; I² = 53%, 39 trials, 1656 participants). The effect of pharmacological placebos versus wait-list or no-treatment on benefits was SMD -0.14 (95% CI -0.39 to 0.11, 9 trials, 279 participants) and the effect of physical placebos was SMD -0.21 (95% CI -0.35 to -0.08, I² = 0%, 17 trials, 896 participants). We found large variations in effect sizes in the psychological and pharmacological placebo comparisons. For specific mental health disorders, we found significant differences in favour of all placebos for sleep-wake disorders, major depressive disorder, and anxiety disorders, but the analyses were imprecise due to sparse data. We found no significant differences in harms for any of the comparisons but the analyses suffered from sparse data. When using a fixed-effect model in a sensitivity analysis on the comparison for usual care versus wait-list and no-treatment, the results were significant with an SMD of -0.46 (95 % CI -0.64 to -0.28). We reported an alternative risk of bias model where we excluded the blinding domains seeing how issues with blinding may be seen as part of the review investigation itself. However, this did not markedly change the overall risk of bias profile as most of the trials still included one or more unclear bias domains. AUTHORS' CONCLUSIONS: We found marked variations in effects between placebo versus no-treatment and wait-list and between subtypes of placebo with the same comparisons. Almost all the trials were small with considerable methodological and clinical variability in factors such as mental health population, contents of the included control interventions, and outcome domains. All trials were assessed as high risk of bias and the evidence quality was low to very low. When researchers decide to use placebos or usual care control interventions in trials with people with mental health disorders it will often lead to lower estimated effects of the experimental intervention than when using wait-list or no-treatment controls. The choice of a control intervention therefore has considerable impact on how effective a mental health treatment appears to be. Methodological guideline development is needed to reach a consensus on future standards for the design and reporting of control interventions in mental health intervention research.

Interleukin-1 blocking agents for treating COVID-19.

BACKGROUND: Interleukin-1 (IL-1) blocking agents have been used for treating severe coronavirus disease 2019 (COVID-19), on the premise that their immunomodulatory effect might be beneficial in people with COVID-19. OBJECTIVES: To assess the effects of IL-1 blocking agents compared with standard care alone or with placebo on effectiveness and safety outcomes in people with COVID-19. We will update this assessment regularly. SEARCH METHODS: We searched the Cochrane COVID-19 Study Register and the COVID-19 L-OVE Platform (search date 5 November 2021). These sources are maintained through regular searches of MEDLINE, Embase, CENTRAL, trial registers and other sources. We also checked the World Health Organization International Clinical Trials Registry Platform, regulatory agency websites, Retraction Watch (search date 3 November 2021). SELECTION CRITERIA: We included randomised controlled trials (RCTs) evaluating IL-1 blocking agents compared with standard care alone or with placebo for people with COVID-19, regardless of disease severity. DATA COLLECTION AND ANALYSIS: We followed Cochrane methodology. The protocol was amended to reduce the number of outcomes considered. Two researchers independently screened and extracted data and assessed the risk of bias with the Cochrane Risk of Bias 2 tool. We rated the certainty of evidence using the GRADE approach for the critical outcomes of clinical improvement (Day 28; ≥ D60); WHO Clinical Progression Score of level 7 or above (i.e. the proportion of participants with mechanical ventilation +/- additional organ support OR death) (D28; ≥ D60); all-cause mortality (D28; ≥ D60); incidence of any adverse events; and incidence of serious adverse events. MAIN RESULTS: We identified four RCTs of anakinra (three published in peer-reviewed journals, one reported as a preprint) and two RCTs of canakinumab (published in peer-reviewed journals). All trials were multicentre (2 to 133 centres). Two trials stopped early (one due to futility and one as the trigger for inferiority was met). The median/mean age range varied from 58 to 68 years; the proportion of men varied from 58% to 77%. All participants were hospitalised; 67% to 100% were on oxygen at baseline but not intubated; between 0% and 33% were intubated at baseline. We identified a further 16 registered trials with no results available, of which 15 assessed anakinra (four completed, four terminated, five ongoing, three not recruiting) and one (completed) trial assessed canakinumab. Effectiveness of anakinra for people with COVID-19 Anakinra probably results in little or no increase in clinical improvement at D28 (risk ratio (RR) 1.08, 95% confidence interval (CI) 0.97 to 1.20; 3 RCTs, 837 participants; absolute effect: 59 more per 1000 (from 22 fewer to 147 more); moderate-certainty evidence. The evidence is uncertain about an effect of anakinra on 1) the proportion of participants with a WHO Clinical Progression Score of level 7 or above at D28 (RR 0.67, 95% CI 0.36 to 1.22; 2 RCTs, 722 participants; absolute effect: 55 fewer per 1000 (from 107 fewer to 37 more); low-certainty evidence) and ≥ D60 (RR 0.54, 95% CI 0.30 to 0.96; 1 RCT, 606 participants; absolute effect: 47 fewer per 1000 (from 72 fewer to 4 fewer) low-certainty evidence); and 2) all-cause mortality at D28 (RR 0.69, 95% CI 0.34 to 1.39; 2 RCTs, 722 participants; absolute effect: 32 fewer per 1000 (from 68 fewer to 40 more); low-certainty evidence).  The evidence is very uncertain about an effect of anakinra on 1) the proportion of participants with clinical improvement at ≥ D60 (RR 0.93, 95% CI 0.78 to 1.12; 1 RCT, 115 participants; absolute effect: 59 fewer per 1000 (from 186 fewer to 102 more); very low-certainty evidence); and 2) all-cause mortality at ≥ D60 (RR 1.03, 95% CI 0.68 to 1.56; 4 RCTs, 1633 participants; absolute effect: 8 more per 1000 (from 84 fewer to 147 more); very low-certainty evidence). Safety of anakinra for people with COVID-19 Anakinra probably results in little or no increase in adverse events (RR 1.02, 95% CI 0.94 to 1.11; 2 RCTs, 722 participants; absolute effect: 14 more per 1000 (from 43 fewer to 78 more); moderate-certainty evidence).  The evidence is uncertain regarding an effect of anakinra on serious adverse events (RR 0.95, 95% CI 0.58 to 1.56; 2 RCTs, 722 participants; absolute effect: 12 fewer per 1000 (from 104 fewer to 138 more); low-certainty evidence). Effectiveness of canakinumab for people with COVID-19 Canakinumab probably results in little or no increase in clinical improvement at D28 (RR 1.05, 95% CI 0.96 to 1.14; 2 RCTs, 499 participants; absolute effect: 42 more per 1000 (from 33 fewer to 116 more); moderate-certainty evidence).  The evidence of an effect of canakinumab is uncertain on 1) the proportion of participants with a WHO Clinical Progression Score of level 7 or above at D28 (RR 0.72, 95% CI 0.44 to 1.20; 2 RCTs, 499 participants; absolute effect: 35 fewer per 1000 (from 69 fewer to 25 more); low-certainty evidence); and 2) all-cause mortality at D28 (RR:0.75; 95% CI 0.39 to 1.42); 2 RCTs, 499 participants; absolute effect: 20 fewer per 1000 (from 48 fewer to 33 more); low-certainty evidence).  The evidence is very uncertain about an effect of canakinumab on all-cause mortality at ≥ D60 (RR 0.55, 95% CI 0.16 to 1.91; 1 RCT, 45 participants; absolute effect: 112 fewer per 1000 (from 210 fewer to 227 more); very low-certainty evidence). Safety of canakinumab for people with COVID-19 Canakinumab probably results in little or no increase in adverse events (RR 1.02; 95% CI 0.86 to 1.21; 1 RCT, 454 participants; absolute effect: 11 more per 1000 (from 74 fewer to 111 more); moderate-certainty evidence). The evidence of an effect of canakinumab on serious adverse events is uncertain (RR 0.80, 95% CI 0.57 to 1.13; 2 RCTs, 499 participants; absolute effect: 44 fewer per 1000 (from 94 fewer to 28 more); low-certainty evidence). AUTHORS' CONCLUSIONS: Overall, we did not find evidence for an important beneficial effect of IL-1 blocking agents. The evidence is uncertain or very uncertain for several outcomes. Sixteen trials of anakinra and canakinumab with no results are currently registered, of which four are completed, and four terminated. The findings of this review are updated on the COVID-NMA platform (covid-nma.com).

[The PRISMA 2020 statement: an updated guideline for reporting systematic reviewsDeclaración PRISMA 2020: una guía actualizada para la publicación de revisiones sistemáticas]. / A declaração PRISMA 2020: diretriz atualizada para relatar revisões sistemáticas.

The Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) statement, published in 2009, was designed to help systematic reviewers transparently report why the review was done, what the authors did, and what they found. Over the past decade, advances in systematic review methodology and terminology have necessitated an update to the guideline. The PRISMA 2020 statement replaces the 2009 statement and includes new reporting guidance that reflects advances in methods to identify, select, appraise, and synthesise studies. The structure and presentation of the items have been modified to facilitate implementation. In this article, we present the PRISMA 2020 27-item checklist, an expanded checklist that details reporting recommendations for each item, the PRISMA 2020 abstract checklist, and the revised flow diagrams for original and updated reviews.

La declaración PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses), publicada en 2009, se diseñó para ayudar a los autores de revisiones sistemáticas a documentar de manera transparente el porqué de la revisión, qué hicieron los autores y qué encontraron. Durante la última década, ha habido muchos avances en la metodología y terminología de las revisiones sistemáticas, lo que ha requerido una actualización de esta guía. La declaración PRISMA 2020 sustituye a la declaración de 2009 e incluye una nueva guía de presentación de las publicaciones que refleja los avances en los métodos para identificar, seleccionar, evaluar y sintetizar estudios. La estructura y la presentación de los ítems ha sido modificada para facilitar su implementación. En este artículo, presentamos la lista de verificación PRISMA 2020 con 27 ítems, y una lista de verificación ampliada que detalla las recomendaciones en la publicación de cada ítem, la lista de verificación del resumen estructurado PRISMA 2020 y el diagrama de flujo revisado para revisiones sistemáticas.

Conflict of Interest Policies at Medical Schools and Teaching Hospitals: A Systematic Review of Cross-sectional Studies.

BACKGROUND: This systematic review aims to estimate the proportion of medical schools and teaching hospitals with conflicts of interest (COI) policies for health research and education, to describe the provisions included in the policies and their impact on research outputs and educational quality or content. METHODS: Experimental and observational studies reporting at least one of the above mentioned aims were included irrespective of language, publication type or geographical setting. MEDLINE, Scopus, Embase and the Cochrane Methodology Register were searched from inception to March 2020. Methodological study quality was assessed using an amended version of the Joanna Briggs Institute's checklist for prevalence studies. RESULTS: Twenty-two cross-sectional studies were included; all were conducted in high-income countries. Of these, 20 studies estimated the prevalence of COI policies, which ranged from 5% to 100% (median: 85%). Twenty studies assessed the provisions included in COI policies with different assessment methods. Of these, nine analysed the strength of the content of medical schools' COI policies using various assessment tools that looked at a range of policy domains. The mean standardised summary score of policy strength ranged from 2% to 73% (median: 30%), with a low score indicating a weak policy. North American institutions more frequently had COI policies and their content was rated as stronger than policies from European institutions. None of the included studies assessed the impact of COI policies on research outputs or educational quality or content. CONCLUSION: Prevalence of COI policies at medical schools and teaching hospitals varied greatly in high-income countries. No studies estimated the prevalence of policies in low to middle-income countries. The content of COI policies varied widely and while most European institutions ranked poorly, in North America more medical schools had strong policies. No studies were identified on impact of COI policies on research outputs and educational quality or content.